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DERIVING MOTION DETECTION INFORMATION FROM MOTION- 
VECTOR-SEARCH TYPE VIDEO ENCODERS 



Introduction: 

When digital video started becoming a common product, many image processing 
algorithms were implemented to offer a large variety of tools. One of them is Video 
Motion Detection (VMD) - the ability to use digital video for detecting motion in the 
field-of-view. This algorithm provides a motion detection sensor for "free" - when a 
digital surveillance system is installed, the motion detection data is derived from the 
video images without any extra cost. 

Digital video is usually compressed (encoded) before it is distributed. Today's 
methods of video encoding are usually based on Motion- Vector-Search (MVS) based 
video encoders. These algorithms provide high image quality at lower bit-rate, 
enabling distributing the video stream over lower-bandwidth networks. Examples of 
such algorithms are MPEG-2, MPEG-4 and H.264. 

All the image processing algorithms require a substantial amount of computing 
power. Reducing the computing power requirements enables adding performance to 
existing systems and providing the same performance for less money. 
The method described below enables getting 'two for the price of one": on a system 
that uses an MVS-based compression algorithm, it enables getting VMD in almost no 
added processing power. 

Field of Invention: 

This method enables calculating motion detection inside the process of MVS- 
encoding a video stream, using almost no added computing power. 

Prior Art: 
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Video Motion Detection algorithms are well known in the literature. They require 
2xNxM steps for calculating, where N and M are the width and height of the image 
respectively. Systems that utilize them have to pay this price in computing power. 

What's the Problem? (Disadvantages of existing solution): 

The iamount of calculations requires a lot of computing power. 

Description of the Invention: 

The method uses the byproducts of the MVS encoding process to mathematically 
derive motion detection data. Instead of 2xNxM computational steps it uses only 
NxM/256 up to NxM/100. The method was successfully implemented on MPEG-4, 
currently the de-facto standard for streaming video compression. From here on, we 
will use MPEG-4 as the implementation example. 

Drawings: 
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Figure 1: A block diagram of a video streaming application with VMD. 
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Figure 2: A block diagram of a video streaming application with VMD, 
new algorithm used. 
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Figure 3: A block diagram of MPEG-4 data processing flow. 
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Figure 4: A block diagram of the data processing flow of the integrated 
MPEG-4 and VMD. 
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Text: 

Figure 1 shows the regular data flow in a video streaming application: the raw 
, (uncompressed) video images (1) go to both the MPEG-4 (2) and the VMD (4) 
modules. Both modules store a reference image, and both modules make 
calculations on the images. The results (the compressed video stream (3) and 
the VMD result (5)) go to the stream/event handler (6) for decision making 
and to the user. 

Figure 2 shows how the same system works with the new method integrated 
the raw (uncompressed) video images (11) go only to the MPEG-4 module 
(12). The data needed for VMD is gathered and passed to the VMD module 
(17). The size of this data portion is 1/256 of the size of a regular image. The 
VMD module (14) then calculates the VMD result. Both the VMD data (15) 
and the compressed video stream (13) pass as before to the stream/event 
handler (16). 

The motion estimation process in MPEG-4 encoding tries to estimates the 
amount of motion in every 16x1 6-pixel macroblock relative to the previous 
image. For doing this, it calculates the SAD - Sum of Absolute Differences: 

SAD16(xc,yc,xr,yr) = ^j^.j6lC X c + i,y<H.j - RxrH,yr+j| 
Where C is the current image and R is the previous one. When xc=xr and 
yc=yr the two macroblocks are in the same location. When there is a 
difference, they are not in the same location. 



The encoding process tries to find the best fit in the immediate area of the 
macroblock. When there is no motion, the best SAD is in the same location. 
When there is motion, the best SAD is in another location. The Motion 
Estimation process finds the best match, and then derives the Motion Vector 
(MV), which describes the relocation vector from the current location to the 
new one. The Motion Estimation module does this process for every 
macroblock C(x,y) in the current image, and passes the MVs to the Motion 
Compensation module for further processing. 

Figure 3 shows the internals of the MPEG-4 algorithm: we see the motion 
estimation module (101) calculating the SAD and the MV, and passing them 
on to the motion compensation module (102). The process continues and 
eventually the encoded video stream (103) is created. 

Figure 4 shows the integration of the MPEG-4 with the VMD module using 
the proposed method. The only change relative to figure 3 is the SAD table 
(114): the motion estimation module places the SAD values of every 
macroblock in a designated table, and the table is then processed by the VMD 
module (115), to create the VMD data (1 16). 

The VMD module is using the SAD table to derive the amount of motion in 
the whole image,~relative to the previous one. To minimize the noise effect 
this process is accumulated over a number of successive images (currently - 
10) and a weight function is added to emphasize the presence of large objects 
and minimize the effect of small isolated ones. 



# 

Advantages of the Invention over Prior Art: 

Since the above algorithm saves processing time, it enables using less powerful (and 
cheaper) processors for the same task. For existing MVS-based applications it enables 
adding a motion detection feature for hardly no added development and processing 
power. 

Innovative Steps: 

The use MVS-based interim results for calculating VMD data, instead of a full 
implementation of a VMD algorithm. The required amount of calculations are 
between NxM/256 and NxM/100, instead of 2xNxM. 
Glossary of Words and Acronyms: 

MPEG-4: Moving Pictures Experts Group standard for encoding and decoding 
motion video. 

VMD: Video Motion Detection. 

Pixel: the smallest unit in a digital image. 

Macroblock: a rectangular block in an image. 

References: 

The MPEG-4 Book, Fernando Pereira and Touradj Ebrahimi, ed. IMSC press, 2002. 
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CLAIMS 

1 . A method for simultaneously supporting of video motion detection and video 
compression, said method including the steps of: 

providing a digital stream of video frames to the encoder; 

extracting SAD table data by the encoder based on a comparison of frames; 

performing video motion detection analysis based on the extracted SAD table 

data; 

encoding video frame data of the digital stream by the encoder essentially 
concurrently while performing said extracting step; and 

providing the encoded video frame data and video motion analysis to a 
stream/event handler. 

2. A method according to claim 1 wherein said encoder is a motion vector 
search based encoder. 

3. A method according to claim 2 is an encoder chosen from the group of 
encoders which include MPEG-2, MPEG-3, MPEG-4 and H. 264 encoders. 
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Background Questions For The Inventor(s) 



(NEEDED TO FILL OUT THE REQUIRED FORMS) 

The INVENTORS are defined as the individuals that provided a SOLUTION to a 
specific PROBLEM. It is preferable that the leading inventor will answer the 
questions. 

I. General Questions About Inventors: 

1) Who is/are the inventor(s)? For each inventor, give full name, work 
location, telephone number, email address, the date the inventor started to 
work at SERVISION (at least month and year), home address, home mailing 
address (if different), country of citizenship (or countries of citizenship, if 
there are more than one), and manager's name. 



Name of Inventor 


1) Dror Heller 


2) Yosef Rotman 


Work Location 


SerVision 


SerVision 


Work Number 






Work E-mail 


dheller@servision.net 


yosefr@servision.net 


Start Date at SERVISION 


January 2003 


March 2002 


Home & mailing address 


14 November 29 th , 
Jerusalem, ISRAEL 


23/1 Shlomo St. 
Jerusalem, ISRAEL 


Country of Citizenship 


Israel 


Israel 


Manager's 


Gidon Tahan 


Gidon Tahan 



2) If there are more than one inventors, what did each inventor contribute to 
the invention? 

Inventor 1: Design and implementation 
Inventor 2: Ideas and consulting. 



3) Are there inventors who do not work at SERVISION today? If so: 

a) Who are they? 

b) Where do they work now? 
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c) Did they at work at SERVISION at any time during the conception or 
implementation of the invention? 

d) If so, what are the dates that they started and stopped working at 
SERVISION? 

II. History of the Invention : 

[The idea here is to let the Attorneys know how tire idea developed and at what 
stage of development the idea is today. The questions below should be 
answered very briefly.] 

1) What is the current stage of the idea? Select one: Concept ; 

Analysis ; Design ; Prototype ; Bench Test/Alpha Test 

; Pilot Run/Beta Test X Commercial Production 



2) Conception of the idea: 

a) When did the inventors get the idea for the invention (approximately, if 
the exact date is not known)? June 2003 

b) Where did this happen? SerVision 

c) Where there any non-inventors present when the idea was created? No 

d) If so, who? 

3) First sketch or drawing: August 2003 

a) When was the first sketch or drawing of the idea made? SerVision 

b) Who made it? Dror Heller 

c) If you have a copy of the first sketch or drawing, please attach it. 

4) First model or prototype: August 2003 

a) Was a model or prototype of the idea made? Yes 

b) If^so, when was the model or prototype completed? Sept 2003 

c) If this is being done now, when do you expect to complete the model or 
-prototype? 
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5) Alpha testing: 

a) Was the idea alpha tested? Yes 

a) Who performed the alpha test? Dror Heller 

b) When was the idea alpha tested? August 2003 

c) Apart from the inventors, who else was present during the alpha testing 
of the idea? None 

d) Where was the idea alpha tested? SerVision 

6) Beta testing (at a customer site or partner site): 
a) Was the idea beta tested? No 

e) Who performed the beta test? 

f) When was the idea beta tested? 

g) Apart from the inventors, who else was present during the beta testing of 
the idea? 

h) Where was the idea beta tested? 

7) Has the idea been produced commercially? If so, when, how many units 
were produced (approximately), by who (that is, who was the manufacturer), 
and for who (that is, who was the customer)? No 

8) Does this invention impact the project you are working on? Yes. If so: 

a) How does the invention impact on your project? It enables adding VMD 
with almost no added computational power. 

b) How would you categorize the amount of the impact on your project? 

VITAL [ ] IMPORTANT [X], or HELPFUL [ ] 
(REMEMBER, A PATENT DOES NOT NEED TO BE AN INVENTION OF A TOTALLY 
NEW INDUSTRY. IT SIMPLY NEEDS TO BE SOMETHING NEW THAT HAS SOME 
TECHNICAL OR COMMERCIAL VALUE. SMALL OR MODERATE IMPROVEMENTS 
MAKE UP THE VAST MAJORITY OF ALL PATENTS, AND HIGHLY REGARDED BY 
SERVISION.) 

c) Why did you pick that category for the impact on your project? 
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Adding a full VMD algorithm would take cmoputational power the 
system does not have. So any other algorithm will degrade the system's 
performance. 

9) Apart from any impact the invention may have on your project, does this 
invention impact the Company's technology in general? No. If so: 

a) How does the invention impact on your project? 

It saves development time and adds features not without too much 
development effort. 

b) How would you categorize the amount of the impact on your project? 
VITAL [ ] IMPORTANT [X], or HELPFUL [ ] 

(AGAIN, THE GREAT MAJORITY OF PATENTS ARE SMALL OR MODERATE 
IMPROVEMENTS, WITH SOME TECHNICAL OR COMMERCIAL VALUE. ALL THESE 
PA TENTS ARE RESPECTED AND REWARDED BY SERVISION.) 
c) Why did you pick that category for the impact on the Company's technology? 

10) Prior Practice: 

a) Was the invention practiced before in SERVISION? If so, describe 

the circumstances? No 

b) Was the invention practiced before at some place other than 
elsewhere? If so, describe the circumstances? No 

c) Have you seen this solution in writing in the professional media? No 

d) Have you performed a patent search? No If so, did you find any 
patents that were relevant to the invention (even if they werent 
exactly the same)? If so, what the numbers of those patents? 
(Attach copies of whatever relevant patents you have.) Even if you 
have not performed a patent search, have you seen this idea 
described in a different patent? 

e) Have you seen a similar idea described anywhere else? No If so, 
under what circumstances? (That is, a competitor's product, an 
advertisement, a trade show, etc.) 

f) Do you have regular access to trade magazines, technical articles? Do visit 
trade shows, or do you get trade show information from other people? Yes 

g) Where did you get the idea? At work 
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III. Contacts with Outside Parties ; 

1) Up to the date you fill out this form, did you or anyone else you know of 
ever discuss the idea of the invention or the invention itself with anyone 
outside of SERVISION? No. If so: 

a) With whom outside of SERVISION? 

b) When? 

c) Where? 

d) What were the circumstances? (Discussion of idea, or product 
demonstration, or market research, or testing, or joint development, or offer 
to sell, or sale, etc.) 

e) Were samples supplied? 

f) Were written drawings or diagrams supplied? 

g) At the time of each such contact with an outside party, did SERVISION 
have a Non-Disclosure Agreement between SERVISION and the party? If so, 
do you have this Agreement or do you know who does have the Agreement? 
(If you have it, please attach a copy.) 

2) Did you or anyone else at SERVISION make an oral or written offer to 
sell? If so, please describe this offer, including name of potential customer, 
price offered, result of the offer, etc. No 

3) Do you or anyone you know of plan to discuss the idea of the invention or 
the invention itself with anyone outside of SERVISION within the next six (6) 
months? If so, what will be the circumstances of this discussion? (Again, 
include any planned discussion, demonstration, market research, testing, joint 
development, offer to sell, intent to sell, etc.) No 

4) Up to the date you fill out this form, was the idea ever published publicly? 
No. Does the idea appear in any SERVISION promotional literature? Does 
the idea appear in any article or paper that was published? Was the idea 
ever presented at a trade show? 
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5) Up to the date you fill out this form, was there ever any other public 
announcement or other revelation of the idea of the invention or the 
invention itself? No If so, when and under what circumstances? (An article, a 
trade show, a meeting, etc.) 

6) Do you know if anyone is planning any public announcement or other 
public revelation of the idea of the invention or the invention itself over the 
next six (6) months? No If so, when and under what circumstances? 
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